Abstract

This document presents a brief overview of the current state of the Intended Learning Objectives (hereafter ILO). Including information on:

  • The number of ILOs in our courses.

  • The orientation of language used in ILOs

  • The most common capacities the ILOs promote broken down by Level and Concentration

  • [In progress] A description of how the current objectives map to Bloom’s Taxonomy of learning.

  • [NOT DONE YET] Recommendations on how to write ILOs

  • [NOT DONE YET] Suggested rephrasing of current ILO’s

A brief overview of our data

For this analysis I used the ILOs for the year 2018/2019 provided by Edith. The ILOs of the projects (PRO) were missing from this file, and have therefore not been included in the subsequent analysis. (I did have the ILOs for the Undergraduate Research Projects (UGR), so these are inlcuded).

The data contained a total of 142 courses, which amounted to 625 ILOs. All quotes from ILOs are referenced with base to the ILOs_2018_2019.docx document.

(It is possible to add some insight into the ILOs of projects and Capstone by using data from previous years. However, this old data does not include any of the ILOs for the new research projects, nor some of the ILOs for older projects, such as Deep Reading. The file is called “Missing_ILOs.xlsx” and is available for integration in the present report upon request).

How many of ILOs do our courses have? (distribution per course)

In general, the courses had an average of 4.4 ILOs per course. Which were distributed as follows:

A breakdown by Level and Concentration gives:

##With what orientation are they written? (Student vs. Course) It was possible to distinguish two categories in the way ILOs were formulated. ILOs were either as student (S) oriented or course (C) oriented. Student oriented ILOs described what the student was supposed to achieve or have learned during the course, whilst course oriented ILOs described the aims of the course. For example:

  1. Student Oriented ILO:

“To offer a broad overview of scientific models” (COR1005, p. 1)
“To acquaint students with the problems…” (HUM2030, p.3)
“To enhance their research skills” (SKI1009, p.11)

  1. Course Oriented ILO:

“Apply basic bookkeeping techniques” (SSC2022, p.15)
“To reflect on the relevance and utility of social theory in general”(SSC2028, p.15)

A few courses had some ILOs formulated in a student oriented fashion and others ILOs in a course oriented fashion. Thus, overview bellow is done at a granularity of ILO not courses.

In general, this is what we have in the curriculum in raw numbers:

In percentages, we have the following:

###Inspecting NA orientation category The NA category represents courses that I did not classify in any of those categories. They were the following ILOs:

Table of ILOs wihout orientation category
Course ILO
SCI2009 1. Human Cells and functions
SCI2009 2. Functional organization of the body
SCI2009 3. Membrane Physiology
SCI2009 4. Cardiac function and blood pressure control
SCI2009 5. Red blood cell function and gas transport and exchange
SCI2009 6. Pulmonary ventilation and regulation
SCI2009 7. Kidney function, intra-and extracellular compartments
SCI2009 8. Neuronal control
SCI2009 9. Hormonal control
SCI2009 10. Gastrointestinal Physiology
SCI2010 1. Fairness and cooperation - in the chapters on Cooperative Games and Bankruptcy Problems
SCI2010 2. Rationality and Common Knowledge - in the chapters on Extensive Form Games, Normal Form Games, Matrix Games, Repeated Games and Repeated Games with Absorbing States
SCI2010 3. Threats and Manipulations - in the part on Repeated Games
SCI2010 4. Expectations - in the chapters on Normal Form Games, Matrix Games, Repeated Games and Repeated Games with Absorbing States
SCI2010 5. Nonmanipulability - in the chapter on Mecanism Design
SCI2031 1. Cells and humoral factors of the innate and adaptive immune system.
SCI2031 2. Cellular and molecular effector mechanisms of the innate and adaptive immunity during inflammation and infection.
SCI2031 3. The structure and function of primary and secondary lymphoid tissue.
SCI2031 4. The processes in the immune response after immunisation and vaccination.
SCI2031 5. Immune mechanisms in disease.
SSC1009 All previous 5 objectives
SSC2052 NA
SSC3030 1. The legal foundations of the European Union
SSC3030 2. The institutions of the EU, their historical evolution and the horizontal relationship between them
SSC3030 3. The vertical relationship between the EU and the Member States including the principles of conferral, supremacy, subsidiarity, proportionality and loyalty
SSC3030 4. The implementation and enforcement mechanisms of EU law infringement proceedings against Member States, enforcement through national courts, legal review of EU action.
SSC3057 Standard economic paradigm of expected utility theory and rational choice
SSC3057 What is the influence of neurology, psychology and sociology on the economic paradigm
SSC3057 Development of macroeconomics and policies
NA NA

Analysing our ILOs

Methodology

Phase 1- Verb extraction

In order to get a better grasp of the characterization of our ILOs, I manually went through all ILOs and extracted the verb(s) describing what the student was supposed to do in each ILO. This created a distiction between the course verb and the student verb, as well as between the action verb and the intended verb. For instance:

1. Course verb/Student verb:
For the ILO “to provide students with […] perspectives to examine…” (COR1004, p.1) the extracted verb is “[to] examine” not “[to] provide”.

2. Action verb/Intended verb: For the ILO “to have the ability to interpret dynamical phenomena…”(SCI3006, p. 9) the verb “[to] interpret” was extracted not “[to] have (the ability)”.

In cases where the verb used was missleading, descriptive words were included. For instance, in the ILO “Gain basic knowledge in using economic/statistical data and present them in an informative way” (SSC2038, p. 15), the words “Gain basic knowledge” were recorded, as opposed to simply “[to] gain”. For this specific ILO, the verbs “use” and “present” were also extracted, as each ILO could have more than one verb associated to them.

During this phase I tried to keep as close as possible to the original formulation, although as I advanced through the ILOs I started to adapt some of the formulations to create some consistency with previously encountered data. Thus, particularily at the end formulations such as “to perform an analysis” were simply extracted as “analyse”. Moreover, because of this fidelity principle, some of the verbs for ilos were not verbs at all but conveyed the expected outcome. For instance, “overview” was extracted from the following ILO: “To give an overview over the different media platforms and media practices” (HUM2022, p.3), since the student was suppoded to ‘get an overview’.

This phase was performed in Excel and all extracted verbs were recorded in the same row as the original formulation, so it is possible to trace back the work and contest my check my interpretations.

In the following list, it is possible to see all the verbs that were extracted:

Extracted verbs after Phase 1
Verbs
to know
to have knowledge
to understand
to develop attitude
to develop understanding
basic understanding
get overview
work with models
model
acquaint
familiarize
introduce
examine
develop own analysis
[get] notion
understand
evaluate
[get] introduction
explore
test
integrate
analyse
express
[gain] familiarity
to write
[get] overview
highlight
pinpoint characteristics
explain
apply
set up
write
get acquainted
gain insight
basic introduction
explore the meaning
explore how
to study
critical reflection
learn
to trace
reflect
gain understanding
basic knowledge
select
communicate
demonstrate awareness
recognize
view films critically
[get] showned
NA
close reading
identify
develop sensitivity
distinguish
trace
to look at
construct design
address the what, who, why, how, when, where
study
present
grasp
critically analyse
contribute to debates
develop own understaning
describe
take position in debate
gain sufficient background
be presented with
appreciate (more)
acquire basic toolbox
reason qualitatively
be prepared
notice
elaborate
give examples
point out
design
use
improve problem solving skills
solve
develop computational skills
know
gain basic practical knowledge
obtain basic knowledge
review
cast
make use
interpret
presentation skills
meet
collect
demonstrate
translate
establish (link)
enrich discussion
propose (solutions)
work
formulate
produce
interview
execute
conduct
practice
enhance research skills
experience
brain storm
deliver
integrate (visual aids)
give feedback
carve out (underlying structure)
build argument
plan
make coherent
gain expertise
perform
report
find
prepare
position interest
work together
transcribe
be engaged (in scientific inquiry)
gain funcionalist vision
judge
comprehend
differentiate
coduct
make contact
discover
read
construct
engage in socio-legal thinking
be conversant
survey
develop
see
consider
retreive
list
provide (reasons)
form reasoned opinions
become aware
research
discuss
develop (an approach to)
retain
investigate
reflect (critically)
(conduct) research
cope
work in groups
keep infromed
frame
approach
view
assess
appreciate
value
gain perspective
deal with
discern
peer reviewing skills
to be able to see
use general models and modelling techiques
compare
connect
get an idea of
situate in context
debate
to explain
use specialized terms
reconstruct
(apply)adopt method
execute design
contextualize
think practically
talk
reason analyticaly
compose
debug
define
reason academically
inspire
pitch
organise
adapt
avoid
modify
gather
turn into
set up plan
observing
create
understand (critically)
manage
unlock (ongoing debates)
speak (“orally”)
illustrate
weigh
retain content
extract
give (opinion)
display (tolerance)
use frameworks
incorporate feedback
to put trends in context
synthesise and explain
to discuss
apply criticism
run programs
answer
outline
interact
speak
choose
taking fieldnotes & interviewing
reduce (stereotypes)
write/plead
define/analyse/answer
criticize
to further (research, analyical and writing skills)
reinforce (opinion)
use knowledge

Phase 2- Standardisation

As you can see, some verbs are really similar. For example, we have: “to understand”, “understand” and “basic understanding”. Therefore, in Phase 2 I standardised some of the vocabulary. All of the previous words were replaced by the same words: “understand”. Here is an overview of the replacements:

## [1] "The verbs that were taken to be the same as \"understand\" are: to understand, to develop understanding, basic understanding, understand, gain understanding, develop own understaning, understand (critically)"
## [1] "The verbs that were taken to be the same as \"know\" are: to know, to have knowledge, basic knowledge, know, obtain basic knowledge"
## [1] "The verbs that were taken to be the same as \"[gain] familiarity\" are: familiarize, [gain] familiarity"
## [1] "The verbs that were taken to be the same as \"analyse\" are: develop own analysis, analyse, critically analyse"
## [1] "The verbs that were taken to be the same as \"[be] introduced\" are: introduce, [get] introduction, basic introduction"
## [1] "The verbs that were taken to be the same as \"[get] overview\" are: get overview, [get] overview"
## [1] "The verbs that were taken to be the same as \"[get] acquainted\" are: acquaint, get acquainted"

The result is a table like this:

Table of ILOs with extravted standardized verb
Course ILO Verb
COR1002 To have knowledge of a number of specific problems in the foundations of the social sciences, such as explanation vs understanding, structuralist vs individualist views on explaining human behavior, the limits of prediction in the social sciences, the role of social science in society know
COR1002 To have knowledge of the major problems or topics in the philosophy of science, such as the demarcation between science and non-science, the role of observation in science, the nature of scientific method notably induction and falsification, the issues of realism and instrumentalism, the problem of progress in knowledgethe difficulties regarding causality in history know
COR1002 To know the major approaches in the philosophy of science, such as the traditional or received view, Karl Poppers critical rationalism and its variants, Kuhns theory of scientific revolutions know
COR1003 To Develop a critical understanding concerning the relation between perspective bias, facts, and context, as well as the difficulties regarding causality in history. understand
COR1003 To develop a critical attitude towards the interpretation of historical data and processes develop attitude
COR1003 To understand the main trends in politics, demography, society and culture since 1945, and will be able to see and put these trends in a global context. understand
COR1003 To understand the main trends in politics, demography, society and culture since 1945, and will be able to see and put these trends in a global context. be able to see
COR1003 To understand the main trends in politics, demography, society and culture since 1945, and will be able to see and put these trends in a global context. put trends in context
COR1004 To provide the students with a basic understanding of what political philosophy is about and why it is important and useful in understanding contemporary democratic societies. understand
COR1004 To understand the central concepts like justice and equality in theory, and in application. understand
COR1005 To offer a broad overview of scientific models and modelling techniques in different disciplines [get] overview
COR1005 To teach students how to model a specific phenomenon by using general models and modelling techniques model
COR1005 To teach students how to model a specific phenomenon by using general models and modelling techniques use general models and modelling techiques

Results

Rough numeric overview:

After the previous methodology has been applied, we have 771 individual capacity learning objectives (hereafter capacities). These are the abilities we intend to promote in our students according to our learning objectives and correspond to individual “verbs” (e.g. “understand”, “analyse”). The breakdown by level is as follows:

Level 1000: 123 capacities

Level 2000: 387 capacities.

Level 3000: 123 capacities.

Graphically:
Across all courses:

By Level:

Capacities inspection

So what are these capacities?

Across all courses

Breakdown by Level

Breakdown by Concentration

#### Breakdown by Cluster #Mapping to Blooms taxonomy ##Methodology Objective: To map the verbs used in the ILOs to the cognitive process dimension of Bloom’s Taxonomy.

Procedure:

  1. The verbs the previoulsy identified and standardised verbs from the ILOs were extracted and put into a separate Excel file.

  2. The verb was matched to a dimension on and mapped to Blooms dimensions using Bloom’s Taxonomy Verb Chart form the University of Arkansas (available here: https://tips.uark.edu/blooms-taxonomy-verb-chart/) as a guide.

  3. The file was imported back and the outcome can be seen below.

Brief description of Bloom’s Taxonomy

In general learning objectives are understood as consisting of two parts, usually a verb and a noun, corresponding to the

“the kind of behavior to be developed in the student and the content . .. in which this behavior is to operate”(Tyler,1949, p. 30).

Roughly, in Bloom’s Taxonomy the behaviour or verb is refered to as the cognitive process. This is to promote an understandning of the verb in the learning objectives in terms of the cognitive activity of the student rather than their end behaviour. For instance, a learning objective of the kind “the student will be able to list…” implies (using Bloom) the simple act of remembering. In contrast, without emphasis on cognitive processes, a student might be required to analyse material from several sources and develop the list which they then communicate. This is far more complex than remembering and it is why the emphasis is made on cognitive activities to avoid ambiguity.

The levels of the congnitive process dimension in Bloom’s taxonomy are:

  1. Remember - Retreive knowledge from long term memory.

  2. Understand - Construct meaning from materials.

  3. Apply - Carry out or use a procedure in a given situation.

  4. Analyse - Break material into constituent parts, and determine the relation between parts to one another and to an overall structure or purpose

  5. Evaluate - Make judgments based on criteria and standards

  6. Create - Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure.

(Anderson, Krathwohl & Bloom, 2001, “5.1 The cognitive process dimension”)

Results

Mapping the verbs used according to the capacity, we have:

Table of verbs and their corresponding cognitive dimensions
Verb Remember Understand Apply Analyse Evaluate Create
know 1 1 0 0 0 0
understand 0 1 0 0 0 0
develop attitude 0 0 0 0 0 1
[get] overview 0 1 0 0 0 0
work with models 0 0 1 0 0 0
model 0 0 0 0 0 1
[get] acquainted 0 1 0 0 0 0
[gain] familiarity 0 1 0 0 0 0
[be] introduced 0 1 0 0 0 0
examine 0 0 0 1 0 0
analyse 0 0 0 1 0 0
[get] notion 0 1 0 0 0 0
evaluate 0 0 0 0 1 0
[get] introduction 0 1 0 0 0 0
explore 0 0 1 1 0 0
test 0 0 0 0 1 0
integrate 0 0 0 0 0 1
express 0 1 1 0 0 0
write 0 0 1 0 0 0
highlight 0 0 0 1 0 0
pinpoint characteristics 0 0 0 1 0 0
explain 0 1 0 1 1 0
apply 0 0 1 0 0 0
set up 0 0 1 0 0 0
gain insight 0 1 0 0 0 0
explore the meaning 0 0 0 1 0 0
explore how 0 0 0 1 0 0
study 0 0 0 1 0 0
critical reflection 0 0 0 0 1 0
learn 0 1 1 0 0 0
trace 0 0 1 0 0 0
reflect 0 0 0 0 1 0
select 0 0 0 0 1 0
communicate 0 1 0 0 0 0
demonstrate awareness 0 0 1 0 0 0
recognize 1 0 0 0 0 0
view films critically 0 0 0 1 1 0
[get] showned 1 0 0 0 0 0
NA 0 0 0 0 0 0
close reading 0 0 0 1 0 0
identify 1 0 0 0 0 0
develop sensitivity 0 0 1 0 0 0
distinguish 0 0 0 1 0 0
look at 0 1 0 0 0 0
construct design 0 0 0 0 0 1
address the what, who, why, how, when, where 0 1 0 0 0 0
present 0 0 1 0 0 0
grasp 0 0 1 0 0 0
contribute debates 0 0 0 0 0 0
describe 0 1 0 0 0 0
take position in debate 0 0 1 0 0 0
gain sufficient background 0 1 0 0 0 0
be presented with 0 1 0 0 0 0
appreciate (more) 0 0 0 1 0 0
acquire basic toolbox 0 0 1 0 0 0
reason qualitatively 0 0 0 1 0 0
be prepared 0 1 0 0 0 0
notice 1 0 0 0 0 0
elaborate 0 0 1 0 0 0
give examples 0 1 0 0 0 0
point out 0 1 0 0 0 0
design 0 0 0 0 0 1
use 0 0 1 0 0 0
improve problem solving skills 0 0 1 0 0 0
solve 0 0 1 0 0 0
develop computational skills 0 0 1 0 0 0
gain basic practical knowledge 0 0 1 0 0 0
review 0 1 0 0 0 0
cast 0 0 1 0 0 0
make use 0 0 1 0 0 0
interpret 0 1 0 0 0 0
presentation skills 0 0 1 0 0 0
meet 0 0 1 0 0 0
collect 0 0 1 0 0 0
demonstrate 0 0 0 1 0 0
translate 0 0 1 0 0 0
establish (link) 0 1 0 0 0 0
enrich discussion 0 0 1 0 0 0
propose (solutions) 0 0 1 0 0 0
work 0 0 1 0 0 0
formulate 0 1 0 0 0 0
produce 0 0 0 0 0 1
interview 0 0 1 0 0 0
execute 0 0 1 0 0 0
conduct 0 0 1 0 0 0
practice 0 0 1 0 0 0
enhance research skills 0 0 0 0 0 1
experience 0 0 1 0 0 0
brain storm 1 0 0 0 0 0
deliver 0 0 1 0 0 0
integrate (visual aids) 0 0 1 0 0 0
give feedback 0 0 0 0 1 0
carve out (underlying structure) 0 0 0 1 0 0
build argument 0 0 0 0 0 1
plan 0 0 0 0 0 1
make coherent 0 0 0 0 1 0
gain expertise 0 0 0 1 0 0
perform 0 0 1 0 0 0
report 0 0 0 1 0 0
find 0 0 0 0 1 0
prepare 0 0 1 0 0 0
position interest 0 0 0 0 1 0
work together 0 0 1 0 0 0
transcribe 0 0 0 0 0 0
be engaged (in scientific inquiry) 0 0 1 0 0 0
gain funcionalist vision 0 1 0 0 0 0
judge 0 0 0 0 1 0
comprehend 0 1 0 0 0 0
differentiate 0 0 0 0 1 0
coduct 0 0 1 0 0 0
make contact 0 0 1 0 0 0
discover 0 0 1 0 0 0
read 0 1 0 0 0 0
construct 0 0 0 0 0 1
engage in socio-legal thinking 0 0 0 0 1 0
be conversant 0 1 0 0 0 0
survey 0 1 0 0 0 0
develop 0 0 0 0 0 1
see 0 1 0 0 0 0
consider 0 0 0 1 0 0
retreive 1 0 0 0 0 0
list 1 0 0 0 0 0
provide (reasons) 0 1 0 0 0 0
form reasoned opinions 0 0 0 0 1 0
become aware 0 1 0 0 0 0
research 0 0 0 0 0 1
discuss 0 0 0 1 0 0
develop (an approach to) 0 0 0 0 0 1
retain 1 0 0 0 0 0
investigate 0 0 0 0 1 0
reflect (critically) 0 0 0 0 1 0
(conduct) research 0 0 0 0 0 1
cope 0 0 0 1 0 0
work in groups 0 0 1 0 0 0
keep infromed 1 0 0 0 0 0
frame 0 0 0 1 0 0
approach 0 0 1 0 0 0
view 0 1 0 0 0 0
assess 0 0 0 0 1 0
appreciate 0 0 0 0 1 0
value 0 0 0 0 1 0
gain perspective 0 1 0 0 0 0
deal with 0 0 1 0 0 0
discern 0 0 0 1 0 0
peer reviewing skills 0 0 0 0 1 0
be able to see 0 1 0 0 0 0
use general models and modelling techiques 0 0 1 0 0 0
compare 0 0 1 0 0 0
connect 0 1 0 0 0 0
get an idea of 0 1 0 0 0 0
situate in context 0 0 0 0 1 0
debate 0 0 1 0 0 0
use specialized terms 0 0 0 1 0 0
reconstruct 0 0 0 0 1 0
(apply)adopt method 0 0 1 0 0 0
execute design 0 0 1 0 0 0
contextualize 0 0 0 0 1 0
think practically 0 0 0 0 1 0
talk 0 1 0 0 0 0
reason analyticaly 0 0 0 1 0 0
compose 0 0 0 0 0 1
debug 0 0 1 0 0 0
define 1 0 0 0 0 0
reason academically 0 0 0 1 0 0
inspire 0 1 0 0 0 0
pitch 0 0 1 0 0 0
organise 0 0 1 0 0 0
adapt 0 0 0 0 1 0
avoid 0 0 1 0 0 0
modify 0 0 0 0 1 0
gather 1 0 0 0 0 0
turn into 0 0 1 0 0 0
set up plan 0 0 1 0 0 0
observing 0 1 0 0 0 0
create 0 0 0 0 0 1
understand (critically) 0 1 0 0 0 0
manage 0 0 1 0 0 0
unlock (ongoing debates) 0 1 0 0 0 0
speak (“orally”) 0 0 1 0 0 0
illustrate 0 1 1 0 0 0
weigh 0 0 0 0 1 0
retain content 1 0 0 0 0 0
extract 0 0 1 0 0 0
give (opinion) 0 1 0 0 0 0
display (tolerance) 0 0 1 0 0 0
use frameworks 0 0 1 0 0 0
incorporate feedback 0 0 1 0 0 0
put trends in context 0 0 0 0 1 0
synthesise and explain 0 1 0 0 0 0
apply criticism 0 0 1 0 0 0
run programs 0 0 1 0 0 0
answer 0 1 0 0 0 0
outline 0 1 0 0 0 0
interact 0 0 1 0 0 0
speak 0 0 1 0 0 0
choose 0 0 0 0 1 0
taking fieldnotes & interviewing 0 0 1 0 0 0
reduce (stereotypes) 0 0 0 0 1 0
write/plead 0 0 0 0 0 1
define/analyse/answer 0 0 0 1 0 0
criticize 0 0 0 0 1 0
further (research, analyical and writing skills) 0 0 0 0 0 1
reinforce (opinion) 0 1 0 0 0 0
use knowledge 0 0 1 0 0 0

Graphically

Across all courses

Breakdown by Level

Breakdown by Concentration

#### Breakdown by Cluster Breakdown by cluster and level (levels appear as 1 = 1000, 2 = 2000, 3 = 3000) ## Quality of ILOs According to the UM framework of Constructive Alignment at Course Level, “an ILO is well-formulated when a student who reads the ILO knows what to do and how well to do it in order to achieve the ILO.” (https://constructivealignment.maastrichtuniversity.nl/ilos-course-level/) For which an ILO must be: 1) Fromulated from the student’s perspective 2) Clarify the type of knowledge 3) Clarify the topics that will be taught/learned 4) Clarify the level of understanding or performance (and how this achievement is displayed) of each chosen topic.

Taking this into account the following criteria were developed to evaluate the quality of ILOs:

  1. Formulated from student perspective (0,1):
  2. Concise (-1, 0 , 1) The ILO has a single cognitive process or a single knowledge dimension, or there is a clear strong relation between them.
  3. Clear Capacity (-1, 0 , 1) Clear what student needs to do, (ideally it is measurable/observable), it should be fairly easy to move from this to assessment
  4. Clear Content (-1, 0 , 1) Content is clear and has specific terminology that could be identified by someone in the field who does not teach the course.
  5. Measurable behaviour (0, 1) The cognitive process dimension is observable. [TO BE FINISHED]
  6. Phrasing (-1, 0 , 1) Clear concise and simple phrasing with no spelling or grammatical errors.

Note:Clear Capacity was separated from measurable behaviour to allow for verbs such as “understand” to be included in clear capacity, however it is not obvious how moves from there to assessment, which would be if observable behaviours where used.

Course ILO ranking Concise Clear_capacity Clear_content Phrasing Student_perspective
HUM1012 To test this theory on songs as well as poems 5 1 1 1 1 1
HUM1012 To integrate gender and other axes of difference into the study of the lyric 5 1 1 1 1 1
HUM1012 To express your ideas about poems and songs 5 1 1 1 1 1
HUM1012 To write a poem or song, thereby experiencing the creative process that is involved first-hand 5 1 1 1 1 1
HUM1014 pinpoint key characteristics of movements and developments in western literature since the 1850s, such as realism, modernism, and post-modernism. 5 1 1 1 1 1
HUM1014 explain/understand several key principles and concepts of literary studies, including the concepts narrative, author, character, reader, plot, exposition, causal events, climax, denouement, narration, focalization. 5 1 1 1 1 1
HUM1014 Set up a small research project within the field of literary studies under supervision of the tutor 5 1 1 1 1 1
HUM1014 write a (comparative) research paper about literature 5 1 1 1 1 1
HUM2013 3. To learn how to write an art review. 5 1 1 1 1 1
HUM2014 2. To know and understand what is meant by modern and postmodern, and be able discuss this in relation to various philosophical positions dealt with in the course 5 1 1 1 1 1
HUM2014 3. To be able to trace and explain similarities and crucial differences between the various philosophical positions discussed 5 1 1 1 1 1
HUM2014 5. Be able to evaluate the theories and ideas in critical arguments. 5 1 1 1 1 1
HUM2031 3. To select the appropriate theories and methodological tools for analysis that best suits the material and argument. 5 1 1 1 1 1
HUM2031 6. To recognize the interdisciplinary nature of visual culture in its historical and contemporary overlap with scientific, artistic, and economic imaginaries. 5 1 1 1 1 1
HUM2043 To learn to write a critical and informed film analysis essay. 5 1 1 1 1 1
HUM2046 To explain the differences between technological determinism and the Science and Technology Studies STS approach 5 1 1 1 1 1
HUM2046 To explain the methodological approaches in STS such as SCOT and Actor-Network Theory 5 1 1 1 1 1
HUM2056 To analyze debates connected to contested memorial monuments, competing victim memories and “multidirectional memory” (Rothberg) 5 1 1 1 1 1
HUM3040 3. To explain how multiple identities and experiences of difference and inequality interact by adopting intersectionality as a critical theory and method. 5 1 1 1 1 1
HUM3040 4. To apply the analytical and critical skills needed to examine the dynamics through which identity and difference, inclusion and exclusion, equality and inequality are continuously produced and reproduced. 5 1 1 1 1 1

Examples of Rephrasing ILOs

Bellow some examples of how ILOs could be rephrased:

Course ILO ILO_new Changed
COR1002 To know the major approaches in the philosophy of science, such as the traditional or received view, Karl Poppers critical rationalism and its variants, Kuhns theory of scientific revolutions explain and compare the major approaches in the philosophy of science, such as the traditional or received view, Karl Poppers critical rationalism and its variants, Kuhns theroy of scientific revolutions. 1
COR1002 To have knowledge of the major problems or topics in the philosophy of science, such as the demarcation between science and non-science, the role of observation in science, the nature of scientific method notably induction and falsification, the issues of realism and instrumentalism, the problem of progress in knowledgethe difficulties regarding causality in history explain and discuss the major problems or topics in the philosophy of science, such as the demarcation between science and non-science, the role of observation in science, the nature of scientific method notably induction and falsification, the issues of realism and instrumentalism, the problem of progress in knowledgethe difficulties regarding causality in history 1
COR1002 To have knowledge of a number of specific problems in the foundations of the social sciences, such as explanation vs understanding, structuralist vs individualist views on explaining human behavior, the limits of prediction in the social sciences, the role of social science in society explain and discuss a number of specific problems in the foundations of the social sciences, such as explanation vs understanding, structuralist vs individualist views on explaining human behavior, the limits of prediction in the social sciences, the role of social science in society 1
COR1002 NA take a position on these issues and perspectives and argue for its adoption 1
COR1003 To understand the main trends in politics, demography, society and culture since 1945, and will be able to see and put these trends in a global context. understand the main trends in politics, demography, society and culture since 1945, and will be able to see and put these trends in a global context. 0
COR1003 To develop a critical attitude towards the interpretation of historical data and processes question and critique the interpretion of historical data and processes 1
COR1003 To Develop a critical understanding concerning the relation between perspective bias, facts, and context, as well as the difficulties regarding causality in history. discuss the relation between perspective bias, facts, and context, as well as the difficulties regarding causality in history. 1
COR1004 To provide the students with a basic understanding of what political philosophy is about and why it is important and useful in understanding contemporary democratic societies. understand what political philosophy is about and use it to examine political debates in contemporary democratic societies. 1
COR1004 To understand the central concepts like justice and equality in theory, and in application. understand the central concepts like justice and equality in theory, and in application. 0
COR1005 To offer a broad overview of scientific models and modelling techniques in different disciplines recall a variety of basic scientific models and basic modelling techniques, and identify the use and purpose of models and modelling techniques in different disciplines. 1
COR1005 To teach students how to work with models in different academic fields use models to understand phenomena in different academic fields and their every-day-life. 1
COR1005 To teach students how to model a specific phenomenon by using general models and modelling techniques model a specific phenomenon using general models and modelling techniques 0
COR1005 NA identify the assumptions of basic models and adapt them to a different context or discipline of interest 1
HUM1003 To acquaint students with some of the foundational texts and formative debates that have shaped Cultural Studies as an academic field of inquiry. discuss foundational texts andformative debates that have shaped Cultural Studies as an academic field of inquiry. 1
HUM1003 To familiarize students with key concepts, themes, and topical debates within contemporary Cultural Studies. explain key concepts, themes and topical debates within contemporary Cultural Studies 1
HUM1003 To introduce students to some of the central theoretical approaches within Cultural Studies, including critical theory, semiotics, material culture studies, gender theory, and critical posthumanism. explain and compare central theoretical approaches within Cultural Studies, including critical theory, semiotics, material culture studies, gender theory, and critical posthumanism in the analysis of culture. 1
HUM1003 To provide students with a variety of interdisciplinary critical perspectives to examine the diverse and often contested meanings of cultural objects and processes. examine diverse and often contested meanings of cultural objects and processes from a cariety of interdisciplinary critical perspectives. 1
HUM1003 To teach students to develop students own analyses of culture through oral and written modes of communication. develop their own analyses of culture through oral and written modes of communication 0
HUM1007 1. To familiarize students with the basic notions required for any serious philosophic discussion follow serious philosophical discussions. 1
HUM1007 2. To familiarize students with the basics of valid and invalid argumentation recognise basic valid and invalid forms of argumentation, construct valid arguments and refute invalid ones. 1
HUM1007 3. To introduce students to some of the decisive developmental moments in the history of philosophy recall some of the decisive developmental moments in the history of philosophy (e.g. Descartes, Hume, Kant), and explain why they were decisive 1
HUM1007 NA identify and explain basic problems from epistemology, logic, axiology, and ethics, recall different solutions, and evaluate said solutions. 1
HUM1010 1. To provide students with a better notion of law as a harmonising phenomenon in European culture identify and explain the harmonising effect of law in European culture 1
HUM1010 2. To provide students with a basic notion of similarities and differences in the approach to law in the various member states of the European Union and the USA compare different approaches to law from the various member states of the European Union and the USA 1
HUM1010 3. To give students a better understanding of basic legal notions such as property, contract and delict explain basic legal notions such as property, contract and delict. 1
HUM1010 4. To provide students with a greater ability to evaluate the significance of the transfer of law making powers from the national to the European institutions evaluate the significance of the transfer of law making powers from the national to the European institutions 0
HUM1011 To provide students with an advanced introduction to the diverse domain of the arts such as painting, literature, music and performance art. NA 0
HUM1011 To broaden the students theoretical understanding of art. NA 0
HUM1012 To explore the theory of the lyric To explore the theory of the lyric 0
HUM1012 To test this theory on songs as well as poems To test this theory on songs as well as poems 0
HUM1012 To integrate gender and other axes of difference into the study of the lyric To integrate gender and other axes of difference into the study of the lyric 0
HUM1012 To be able to analyze songs and poems in a sophisticated way, to discuss lyrical texts and songs systematically analyze songs and poems in a sophisticated way, to discuss lyrical texts and songs systematically 0
HUM1012 To express your ideas about poems and songs To express your ideas about poems and songs 0
HUM1012 To become familiar with a number of classic Anglo-American poems To become familiar with a number of classic Anglo-American poems 0
HUM1012 To know at least one poem or song by heart, and to know how to recite/sing it memorize at least one poem or song by heart, and to know how to recite/sing it 1
HUM1012 To write a poem or song, thereby experiencing the creative process that is involved first-hand To write a poem or song, thereby experiencing the creative process that is involved first-hand 0
HUM1013 1. To provide a basic overview of the development of the idea and identity of Europe. trace the development of the idea and identity of Europe 1
HUM1013 2. Highlighting the specific characteristics of Europe s history, notably in comparison with that of other non-European societies Identify and compare specific characteristics of Europe’s history with those of non-European societies that led to the “Europen miracle” and Europes separate identity. 1
HUM1013 3. Confronting students with some of the basic theoretical and methodological problems in dealing with this subject matter. understand the basic theoretical and methodological problems in dealing with this subject matter 1
HUM1013 NA question and evaluate discourse on Europe, European identity, and identity in general 1
HUM1013 NA judge the advantages and limitations in the study of identity ( including in the social construction of identity, the symbolic construction of community, and the notions of representation, invention of tradition, and “lieux de mémoire”). 1
HUM1014 pinpoint key characteristics of movements and developments in western literature since the 1850s, such as realism, modernism, and post-modernism. pinpoint key characteristics of movements and developments in western literature since the 1850s, such as realism, modernism, and post-modernism. 0
HUM1014 explain/understand several key principles and concepts of literary studies, including the concepts narrative, author, character, reader, plot, exposition, causal events, climax, denouement, narration, focalization. explain/understand several key principles and concepts of literary studies, including the concepts narrative, author, character, reader, plot, exposition, causal events, climax, denouement, narration, focalization. 0
HUM1014  apply several of these literary concepts in the analysis of novels and thus to be able to analyze novels in a sophisticated way and to discuss literature systematically.  apply several of these literary concepts in the analysis of novels and thus to be able to analyze novels in a sophisticated way and to discuss literature systematically. 0
HUM1014 Set up a small research project within the field of literary studies under supervision of the tutor Set up a small research project within the field of literary studies under supervision of the tutor 0
HUM1014 write a (comparative) research paper about literature write a (comparative) research paper about literature 0
HUM2003 To acquaint students with historical configurations and intersections of race, class, gender and sexuality, and the way in which they were conceptualised and sometimes newly invented in science, philosophy and social theory. discuss historical configurations and itersections of race, class, gender and sexuality, and recognise the way in which they were conceptualised and sometimes newly invented in science, philosophy and social theory. 1
HUM2003 To acquaint students with the way in which these configurations of difference have structured cultural texts and images, individual identities and organisations. examine the way in which these configurations of difference structure cultural texts and images, individual identities and organisations. 1
HUM2003 To acquaint students with the way in which such intersecting categories of difference have constituted and still constitute inequalities and differences of power, resulting in invisibility, restricted access to sources etc. examine the way in which such intersecting categories of difference have constituted and still constitute inequalities and differences of power, resulting in invisibility, restricted access to sources etc. 1
HUM2003 To introduce “gender” as a useful category in historical analysis use “gender” as a category in historical analysis. 1
HUM2003 To introduce students to Michel Foucault’s “History of Sexuality” and “discourse analysis” and to introduce students to the “cultural work” of literature within the discourse of colonialism (Joseph Conrad) and to train students to connect theories of gender and diversity to historical case studies connect theories of gender and diversity to historical case studies 1
HUM2003 “” explain the essence of Michel Foucault’s “History of Sexuality,”discourse analysis“, and the”cultural work" of literature within the discourse of colonialism (Joseph Conrad) 1
HUM2003 To get aquainted with the theory of intersectionality use the theory of intersectionality in their analyses of social phenomena 1
HUM2003 NA reflect on how categories of difference play a role in their everyday life 1
HUM2005 1. To provide students with a historical and philosophical overview of these two cultural periods (and worldviews), using an interdisciplinary and topic-oriented approach, based on an opposition of the two above-mentioned periods and views compare the worldviews of enlightment and romanticism, as well as their historical contexts. 1
HUM2005 2. Students should also gain insight in the relationship between social, literary, artistic and philosophical developments in eighteenth and nineteenth century Europe, and get an idea of the far-reaching influence that Romanticism and the Enlightenment, as well as the romantic criticism of the Enlightenment, have had on art, science, and political discussions up to today. explain the relationship between social, literary, artistic and philosophical developments in eighteenth and nineteenth century Europe 1
HUM2005 “” identify the influence that Romanticism and the Enlightment, as well as the romantic criticism of the Enlightment, have had on art, science, and political discussions up to today 1
# Students Profiles Bloom
## # A tibble: 142 x 8
##    Course Concise Clear_capacity Clear_content Phrasing Student_perspec~
##    <chr>    <dbl>          <dbl>         <dbl>    <dbl>            <dbl>
##  1 COR10~       3              0             3        3                3
##  2 COR10~       2              3             5        3                5
##  3 COR10~       0              0             1        2                1
##  4 COR10~       4              2             3        0                0
##  5 HUM10~       5             -1             1        0                0
##  6 HUM10~       3             -3             0        1                0
##  7 HUM10~       4             -1             4        2                0
##  8 HUM10~       2             -2             0        2                0
##  9 HUM10~       5              6             7        8                8
## 10 HUM10~       4             -4             2        4                0
## # ... with 132 more rows, and 2 more variables: ranking <dbl>, n <int>
## # A tibble: 142 x 8
##    Course Concise Clear_capacity Clear_content Phrasing Student_perspec~
##    <chr>    <dbl>          <dbl>         <dbl>    <dbl>            <dbl>
##  1 COR10~   1               0            1        1                  1  
##  2 COR10~   0.4             0.6          1        0.6                1  
##  3 COR10~   0               0            0.5      1                  0.5
##  4 COR10~   1               0.5          0.75     0                  0  
##  5 HUM10~   1              -0.2          0.2      0                  0  
##  6 HUM10~   1              -1            0        0.333              0  
##  7 HUM10~   1              -0.25         1        0.5                0  
##  8 HUM10~   1              -1            0        1                  0  
##  9 HUM10~   0.625           0.75         0.875    1                  1  
## 10 HUM10~   1              -1            0.5      1                  0  
## # ... with 132 more rows, and 2 more variables: ranking <dbl>, n <int>

A very basic sketch of how things are aggregated (here we take every ILO to count equally).

For some courses we do not have the ilos, therefore we do an inner_join

Visualising Students Bloom

Methods in ILO’s

A separate question to consider is: how many of our ILOs explicitly refer to methodology? During the verb extraction, a comment was also made refering to whether or not the ILO explicitly refered to methodology. Out of the 625 ILOs existing in the curricula, there were 15 about methodoloy (this is 2.4%).

Here are the ILO’s which I marked as refering to methodology, the verbs used have been added for completeness:

ILOs concenrning to methodology
Course ILO Verbs_used
HUM1013 3. Confronting students with some of the basic theoretical and methodological problems in dealing with this subject matter. [get] overview
HUM1014 write a (comparative) research paper about literature write, compare
HUM2031 3. To select the appropriate theories and methodological tools for analysis that best suits the material and argument. select, analyse
HUM2046 To explain the methodological approaches in STS such as SCOT and Actor-Network Theory explain
HUM3034 To familiarize students with theory, historiography and methodology of historical sociology, combining a macro historical and a macro sociological approach; [gain] familiarity
HUM3050 To understand and be capable of recognizing different methods that are implemented in aging research, such as visual methods, narrative and biographical methods, and ethnographic approaches. understand, recognize
SCI3046 2. To learn which methods a brain researcher can use to investigate the neuronal bases for different mental processes. learn
SSC1005 To gain a basic understanding of the methods of psychological research. understand
SSC2008 1. Knowledge of general developmental issues with respect to theories and experimental design. know
SSC2050 2. To be able to describe methods and tools typically used in this field and experiments have have been conducted. describe
SSC2059 4. To gain familiarity with methodological tools utilized in the analysis of social movements [gain] familiarity
SSC3002 2. To provide students with a basic understanding of the key concepts, approaches, models and methods of public policy analysis & evaluation. understand
SSC3017 4. Conduct primary research and analyse primary and secondary data in the field of social and environmental entrepreneurship (conduct) research, analyse
UGR2001 4. To select and applying relevant research methodologies accordingly select, apply
UGR3001 4. To select and applying relevant research methodologies accordingly select, apply

Bibliography

Anderson, L., Krathwohl, D., & Bloom, B. (2001). A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives (Complete ed.). New York: Longman.